Multilingual Dependency-based Syntactic and Semantic Parsing

Our CoNLL 2009 Shared Task system includes three cascaded components: syntactic parsing, predicate classification, and semantic role labeling. A pseudo-projective high-order graph-based model is used in our syntactic dependency parser. A support vector machine (SVM) model is used to classify predicate senses. Semantic role labeling is achieved using maximum entropy (MaxEnt) model based semantic role classification and integer linear programming (ILP) based post inference. Finally, we win the first place in the joint task, including both the closed and open challenges. 1 System Architecture Our CoNLL 2009 Shared Task (Hajič et al., 2009): multilingual syntactic and semantic dependencies system includes three cascaded components: syntactic parsing, predicate classification, and semantic role labeling. 2 Syntactic Dependency Parsing We extend our CoNLL 2008 graph-based model (Che et al., 2008) in four ways: 1. We use bigram features to choose multiple possible syntactic labels for one arc, and decide the optimal label during decoding. 2. We extend the model with sibling features (McDonald, 2006). 3. We extend the model with grandchildren features. Rather than only using the left-most and rightmost grandchildren as Carreras (2007) and Johansson and Nugues (2008) did, we use all left and right grandchildren in our model. 4. We adopt the pseudo-projective approach introduced in (Nivre and Nilsson, 2005) to handle the non-projective languages including Czech, German and English. 2.1 Syntactic Label Determining The model of (Che et al., 2008) decided one label for each arc before decoding according to unigram features, which caused lower labeled attachment score (LAS). On the other hand, keeping all possible labels for each arc made the decoding inefficient. Therefore, in the system of this year, we adopt approximate techniques to compromise, as shown in the following formulas. f lbl uni(h, c, l) = f lbl 1 (h, 1, d, l) ∪ f lbl 1 (c, 0, d, l) L1(h, c) = arg max1 l∈L(w · f lbl uni(h, c, l)) f lbl bi (h, c, l) = f lbl 2 (h, c, l) L2(h, c) = arg max2 l∈L1(h,c)(w · {f lbl uni ∪ f lbl bi }) For each arc, we firstly use unigram features to choose the K1-best labels. The second parameter of f lbl 1 (·) indicates whether the node is the head of the arc, and the third parameter indicates the direction. L denotes the whole label set. Then we re-rank the labels by combining the bigram features, and choose K2-best labels. During decoding, we only use the K2 labels chosen for each arc (K2 ¿ K1 < |L|). 2.2 High-order Model and Algorithm Following the Eisner (2000) algorithm, we use spans as the basic unit. A span is defined as a substring of the input sentence whose sub-tree is already produced. Only the start or end words of a span can link with other spans. In this way, the algorithm parses the left and the right dependence of a word independently, and combines them in the later stage. We follow McDonald (2006)’s implementation of first-order Eisner parsing algorithm by modifying its scoring method to incorporate high-order features. Our extended algorithm is shown in Algorithm 1. There are four different span-combining operations. Here we explain two of them that correspond to right-arc (s < t), as shown in Figure 1 and 2. We